Image-Based Modeling of Objects and Human Faces

In this paper, I will provide an overview of our project on 3D object and face modeling from images taken by a free-moving camera. We strive to advance the state of the art in 3D computer vision, and develop flexible and robust techniques for ordinary users to gain 3D experience from a set of casually collected 2D images. Applications include product advertisement on the Web, virtual conference, and interactive games. We briefly cover the following topics: Camera calibration: determine a camera’s internal parameters by just observing a planar pattern under unknown orientations. Stereo rectification: transform a pair of stereo images based on a geometric meaningful criterion such that the epipolar lines of the rectified images are aligned. This will considerably simplify the stereo matching process. Image matching: match points and curves across images within a probabilistic relaxation framework. We can obtain a much richer description of a scene using both points and curves. 3D photo editing: Virtual or real objects can be inserted in the real images, while preserving the correct 3D information of the scene structure. Object modeling: A complete 3D model of an object is built by taking a number of snapshots around the object. Face modeling: With five mouse clicks, a 3D face model is built in a few minutes, and the model can be animated immediately. We have successfully built 3D face models for about one hundred people. Demos on the last three topics will be shown during the conference. Most of the above-described topics are joint work with Zicheng Liu, Charles Loop, and Ying Shan among other colleagues.